User-interaction data collected when a user interacts with content items can be used to create personalized content item ranking models for the user. For example, where the content items are web pages, user-interaction data such as clicks and dwell times can be collected and used to generate a personalized model that can predict which web pages that a user is likely to be interested in from a set of search results. The model can be used to rank or order the web pages in search results for the user.
While such personalized models are useful, there are drawbacks associated with personalized models. One drawback is that personalized models may over-fit user-interaction data onto a ranking of web pages, even where there is no gain over a more global or broadly applicable model. For example, where there is little user-interaction data regarding a particular set of web pages, the personalized model may give too much positive weight to the web pages associated with the user-interaction data in the ranking, and may give too much negative weight to the web pages without any associated user-interaction data in the ranking. The resulting ranking may lead to a poor search experience for the user.